CLIP: A Compact, Load-balancing Index Placement Function

Existing file searching tools do not have the performance or accuracy that search engines have. This is especially a problem in large-scale distributed file systems, where better-performing file searching tools are much needed for enterprise-level systems. Search engines use inverted indices to store terms and other metadata. Although some desktop file searching tools use indices to store file metadata, they are not designed for large-scale distributed file systems. We propose bringing inverted indices to the distributed object-based file system named Ceph via CLIP: a compact load-balancing index placement function that distributes the inverted index across the file system. The inverted index is distributed based on the frequency of terms of the files in the file system.